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Abstract 

Background: There is a need for psychiatric assessment instruments that enable reliable diagnoses in persons with 
hearing loss who have sign language as their primary language. The objective of this study was to assess the 
validity of the Norwegian Sign Language (NSL) version of the Mini International Neuropsychiatric Interview (MINI). 

Methods: The MINI was translated into NSL. Forty^one signing patients consecutively referred to two specialised 
psychiatric units were assessed with a diagnostic interview by clinical experts and with the MINI. Inter-rater reliability 
was assessed with Cohen's kappa and "observed agreement". 

Results: There was 65% agreement between MINI diagnoses and clinical expert diagnoses. Kappa values indicated 
fair to moderate agreement, and observed agreement was above 76% for all diagnoses. The MINI diagnosed more 
co-morbid conditions than did the clinical expert interview (mean diagnoses: 1.9 versus 1.2). Kappa values indicated 
moderate to substantial agreement, and "observed agreement" was above 88%. 

Conclusion: The NSL version performs similarly to other MINI versions and demonstrates adequate reliability and 
validity as a diagnostic instrument for assessing mental disorders in persons who have sign language as their 
primary and preferred language. 
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Background 

Language and communication are fundamental for 
diagnosing most mental disorders. Most assessment 
instruments are designed for use with hearing individuals, 
and many deaf and severely hard-of-hearing people with 
psychiatric illness may receive incorrect diagnoses because 
of challenges in communication between patients and 
professionals [1-3]. The resulting lack of diagnostic 
precision may have serious bearing on treatment adequacy 
and quality. In Norway and other countries, there is a 
need for valid and reliable instruments for assessment of 
mental disorders in deaf and severely hard-of-hearing 
persons who have sign language as their primary and 
preferred language (signers). 
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The Mini International Neuropsychiatric Interview 
(MINI) was developed to meet the need for a brief and 
reliable structured diagnostic interview in clinical practice 
as well as in research [4,5]. In clinical practice, the MINI is 
considered a supplement and not a substitution for regular 
diagnostic intake interviews. It is a structured evaluation 
of most of the major psychiatric conditions [4,5], and 
was therefore selected for translation into Norwegian 
Sign Language (NSL). 

The MINI includes 23 disorders from the tenth revision 
of the International Classification of Diseases and Related 
Health Problems (ICD-10) [6] and from the Diagnostic 
and Statistical Manual of Mental Disorders, Fourth 
Edition (DSM IV) [7]. It is organised in diagnostic 
sections with branching tree logic and two to four 
screening questions with yes or no responses for each 
disorder. Additional symptom questions are asked only 
when a screening question is endorsed [4,8]. Excellent 
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inter-rater and test-retest reliabilities have been reported 
for the English and French versions [9], as well as good to 
very good convergent validity relative to the Composite 
International Diagnostic Interview, CIDI [4] and the 
Structured Clinical Interview for Diagnostic and Statistical 
Manual, SCID [5]. In test-retest analyses, kappa values 
have indicated excellent agreement according to Fleiss 
[10] for six diagnoses, fair to good agreement for six 
diagnoses and poor agreement for seven diagnoses, 
whereas observed inter-rater agreement has been 75% 
or above [5]. 

When a psychiatric diagnostic instrument is translated 
into a new language, reassessment of validity and reliability 
is necessary. Wordings that seem identical in different 
languages may nonetheless be differently interpreted, 
especially when a written text is translated to a sign 
language [11,12]. This is not just a matter of semantic 
divergences, but also of divergences related to cultural 
understanding of concepts, words (signs) and sentences. 
Development of expressions that both convey the question's 
core content and appear meaningful to the patient is vital. 

The MINI has been translated into 43 languages, and 
its validity and reliability have been explored not only 
for the original English and the French versions, but also 
for the Spanish [13], Italian [14], Japanese [15], Moroccan 
[16], Portuguese [17] and Norwegian versions [18] (see 
Table 1). No sign language version has been reported. 
Deaf and hard-of-hearing people have been excluded 
from most mental health research, including valid- 
ation studies of the MINI. Exclusion criteria cited in 
such research include "language problems" [4,14,15], 
"patients who were hearing impaired, not fluent in 
English" [19] and "could not be interviewed due to 
language barriers" [18]. 

Sign languages are natural languages that have evolved 
through use by deaf and hard-of-hearing people. Utterances 
are produced with the hands and face and are visually 
received and decoded [20,21]. Sign languages share 
many linguistic characteristics with spoken languages, 
but also have characteristics that are specific to the 
manual-visual modality [20,21]. Most deaf adults use 
written language and consider it their second language; 
however, a considerable percentage of congenitally deaf 
people struggle with written texts [22,23] . 

To reliably diagnose mental illness in signing persons, 
clinicians must have sufficient sign language competence 
and thorough knowledge of deaf culture. They must also 
have a deep understanding of the potential psychosocial 
consequences of profound hearing loss, the accompanying 
language and communication challenges, and the obstacles 
encountered by many deaf and hard-of-hearing people 
in society. If these requirements are not met, clinicians 
may misinterpret the patient's utterances as symptoms of 
mental illness or thought disturbances, or may overlook 



symptoms of mental illness because they are not commu- 
nicated in the expected manner. 

The present study investigates the functionality of a 
NSL version of the MINI. The main research question is 
whether this version functions in the same manner with 
signing deaf and hard-of-hearing patients as do other 
versions of the MINI with hearing patients. 

Methods 

The present investigation is part of a comprehensive study 
of mental health in individuals referred to specialised 
psychiatric units for deaf and severely hard-of-hearing 
patients, and includes patients using different modes 
of communication and language. 

Participants 

Deaf and severely hard-of-hearing signers who were referred 
to the unit for adults at the National Centre for Mental 
Health and Hearing Impairment, Oslo University Hospital 
and the Regional Centre for Mental Health and Hearing 
Impairment, St. Olav's Hospital, Trondheim were asked to 
participate in the study. Exclusion criteria were age 
below 18 years, spoken language as the main form of 
communication, dual sensory loss requiring tactile 
communication, acute and severe psychiatric or somatic 
illness, and referral for reasons other than assessment of 
mental disorder. Information was given in NSL or other 
forms of visual communication preferred by the person, 
and written consent was obtained from those who decided 
to participate. 

Eighty-eight deaf and severely hard-of-hearing adults 
were referred to the two specialised mental health services 
in 2010 (total count of referrals throughout Norway was 
120 persons). Twenty-eight of the 88 patients did not 
meet the inclusion criteria and five did not appear for 
their appointments. The remaining 55 patients were asked 
to participate in the study; 10 did not consent and four did 
not complete all the assessments. The sample therefore 
comprised 41 patients, or 75% of those who fulfilled the 
inclusion criteria. Table 2 shows background information 
for the participants. There were twice as many female 
as male participants (mean age, 36 years ± 14.1). Most 
participants had less than a college education, and 
one-third held a job or was studying. 

Assessments 

The assessments applied in this study were identical to the 
ordinary diagnostic assessments at intake. The results were 
documented in case notes from which personal identifiers 
had been removed. The diagnoses in these case notes were 
used to validate the diagnoses assessed by the NSL version 
of the MINE 



Table 1 Studies on the validity and reliability of the MINI 



Study & 
language 



Participants 



Validation 



Reliability- 
analysis 



Results, depressive and anxiety disorders 



Concordance; Cohen's kappa mean & range 



Sheehan et al. [5] 
English 



Lecrubier et al. 
[4] French 



Bobes [13] 
Spanish 

Pinninti et al. 
[19] English 

Rossi et al. 
[14] Italian 



Kadri et al. 
[16] Moroccan 



Otsubo et al. 
[1 5] Japanese 



Psychiatric patients 
N = 308 Non-patients, 
N = 62 Total N = 370 



SCID-P' 



Psychiatric patients 
N = 260 Non-patients, 
N = 50 Total N = 310 



Primary health care 
patients N = 1 26 



Psychiatric patients 
N = 1 75 Non-patients, 
N = 50 Total N = 225 



Psychiatric inpatients 
*N = 82 ^N = 169 



CIDK 



Expert 
opinion 



Psychiatric outpatients Expert 
N = 111 opinion 

Psychiatric outpatients 
N = 50 



Expert 
opinion 



''SCID-P' 
Expert 
opinion 



Test-retest and 
inter-rater analyse 



Test-retest and 
inter-rater analyses 



Inter-rater 
analysis 



nter-rater and 
test-retest analyses 



Inter-rater and 
test-retest analyses 



nter-rater and 
test-retest analyses 



Concordance between MINI-CR and SCID-P' diagnoses 
Major depressive disorder kappa:0.84, observed agreement: 92% 

Inter-rater (kappa) 1.00, Retest (kappa) 0.87 Lifetime panic disorder, 
kappa: 0.80, observed agreement: 91% 

Inter-rater (kappa)0.97, Retest (kappa) 0.79 

Concordance between MINI and CIDP diagnoses 

Major depressive disorder (kappa) 0.73, observed agreement: 
87% Test-retest (kappa) 0.83 

Lifetime panic disorder (kappa) 0.68, observed agreement: 
92% Test-retest (kappa) 0.76. 

Inter rater, all diagnoses (kappa) 0.88-1.00 

Major depression, sensitivity 0.94, specificity 0.62 

Generalised anxiety disorder, sensitivity 0.92, specificity 0.65 

Diagnostic agreement in 58% of the cases In 33% the disagreement 
was of substantial nature MINI diagnosed more co-morbid conditions 

Major depressive disorder, current: Inter-rater reliability (kappa) 0.95. 
Test-retest (kappa) 0.45 

Major depressive disorder, recurrent: Inter-rater reliability (kappa) 0.84. 
Test-retest (kappa) 0.35 

Panic disorder lifetime: Inter-rater reliability (kappa) 0.88. Test-retest 
(kappa) 0.49 

Concordance between MINI and expert diagnoses Major depressive 
disorder (kappa) 0.95, observed agreement: 99% 

Social phobia (5% of participants) (kappa) 0.91, obs. agreement: 94% 

Inter-rater and test-retest reliability (kappa) all above 0.80 

Concordance between MINI and SCID-P'^^ expert opinion 

Major depressive disorder, 
kappa: 0.85, obs. agreement: 93%) 

^ kappa: 0.36, obs. agreement: 59%) 
Panic disorder, kappa: 092, obs. agreement: 98%) 

^ kappa: 0.53, obs. agreement: 89%) 
MINI test-retest, kappa: 0.80, obs. agreement: 91% 
MINI Inter-rater, kappa: 0.94, obs. agreement: 97% 



Twenty-two diagnostic categories analysed 

Two diagnoses occurred in less than 5% of 
participants 

Kappa mean: 0.67, range: 0.43-0.90 



Seventeen diagnostic categories analysed 

Four diagnoses occurred in less than 5% of 
participants 

Kappa mean: 0.66, range: 0.35-0.82 



No Cohen's kappa values available 



No Cohen's kappa values available 

Thirty-one diagnoses analysed Inter-rater 
kappa mean: 0.85, range: 0-1.00 

Test-retest kappa mean: 0.41, range: 0-1.00 

nformation on diagnoses occurring in less 
than 5%) of participants not available 

Thirteen diagnostic categories analysed Four 
diagnoses occurred in less than 5%) of participants 

Kappa mean: 0.91, range: 0.79-0.95 



Thirteen diagnostic categories analysed Four 
diagnoses occurred in less than 5% of participants 

Kappa mean: 0.71, range: 0.49-0.93 

^ Eleven diagnostic categories analysed Seven 
diagnoses occurred in less than S%> of participants 

Kappa mean: 0.34, range: 0.03-0.59 



Table 1 Studies on the validity and reliability of the MINI (Continued) 



Marques & 


Primary health care 


SCID^ 




Concordance between MINI and SCID^ diagnoses Depressive 


Eight diagnostic categories analysed Four 


Zuardi [17] 


patients N = 1 20 






disorders, kappa: 0.75, obs. agreement: 92% 


diagnoses occurred in less than 5% of participants 


Portuguese 








Anxiety disorders, kappa: 0.81, obs. agreement: 94% 


Kappa mean: 0.65, range: 0-0.85 


Mordal et al. [18] 


Psychiatric patients, 




Test - retest 


Concordance of MINI diagnoses in test-retest 


Twenty diagnostic categories analysed Six 


Norwegian 


acute ward N = 38 




analysis 




diagnoses occurred in less than 5% of participants 










Major depressive episode, kappa: 0.82, obs. agreement 92% 


Kappa mean: 0.51, range: 0-1.00 










Panic disorder, lifetime, kappa: 0.84, obs. agreement: 92% 





^The Structured Clinical Interview for Diagnostic and Statistical Manual lll-R Patient version. 

^The Composite International Diagnostic Interview. 

^The Structured Clinical Interview for Diagnostic and Statistical Manual IV. 
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Table 2 Participant information (N = 41) 

Number of females 

Age in years mean: 3f 



Decibel loss in the better ear 

Marital status 
Single 

Married and cohabiting 
Divorced, separated & widowed 
Ethnicity 

European/Caucasian 

Asian 

Education 

High school and equivalent 
College degree and above 
Source of income 
Paid work and study loans 
Social welfare and pension 
'gap F at admission 

^GAF S at admission 



29 (71%) 

SD 14.1, median: 34, 
range: 18-83 

mean: 84.9, SD 13.8, median: 90, 
range: 56-1 10 

22 (53.7%) 
1 5 (36.5%) 
4 (9.8%) 

37 (90.2%) 
4 (9.8%) 

30 (73.2%) 
11 (26.8%) 

13 (31.7%) 

28 (68.3%) 

mean: 55.2, SD 12.4, median: 52, 
range: 31-76 

mean: 55.5, SD 1 1 .0, median: 56, 
range: 21-73 



^Global Assessment of Function, Function score. 
^Global Assessment of Function, Symptom score. 



The MINI 

The MINI 5.0.0 was translated from Norwegian into 
NSL in 2008-2009 through an agreement with Dr. Ulrik 
Malt, who was leading the team that translated the 
MINI from English to Norwegian [24]. The translation 
was performed in accordance with internationally ack- 
nowledged translation procedures [25,26] as well as spe- 
cific procedures suggested for translating written and 
spoken material into sign language [11,22,27]. A bilin- 
gual team of hearing and deaf clinicians and researchers 
translated each item into sign language and sent video- 
recordings to deaf and hearing bilingual professionals 
not familiar with the original text, who back-translated 
the items into written Norwegian. The research team 
compared the back-translations to the original written 
text, and if a back-translated item did not correspond 
to the content and intent of the original item, the 
item was discussed and rephrased. Consensus regard- 
ing the original text and back-translations was obtained 
before any item was included in the final video- 
recorded version. The video-recorded version was 
subsequently used to instruct and train the experi- 
enced MINI assessors, who were signers, to pose the 
questions of the MINI through uniform Norwegian 
Sign Language expressions. 



Clinical interview (Expert opinion) 

The standard intake interview used at Oslo University 
Hospital, Clinic of Mental Health and Addiction, was used 
as expert opinion. It was extended to include questions 
about background factors and life events that may affect 
the participants vulnerability to a mental disorder. These 
questions include cause of hearing loss, primary language 
and communication mode, communication environment 
in childhood, educational setting and experiences related 
to growing up with hearing loss. 

Procedure 

The clinical interview was conducted by clinical psycholo- 
gists and psychiatrists, who had long experience with 
deaf and severely hard-of-hearing patients, were skilled 
in sign language, and had extensive knowledge about 
deafness and deaf culture. The communication with the 
participants was in NSL and was adapted to match the 
communication style of the participant as needed. The 
MINI interview was conducted in NSL by experienced 
MINI assessors. 

Each participant was first seen by the team that would 
conduct the clinical intake interview (expert opinion). 
The MINI assessment was completed by another team 
consisting of a deaf therapist who is a native signer who 
posed the questions to the patients, and either a psychiatrist 
or a specialised clinical psychologist who guided the deaf 
co-therapist and the participant through the MINI 
interview and made the diagnostic decisions. Sometimes, 
the latter professional would also ask questions to obtain 
additional information. The two diagnostic teams received 
the referral information but did not share other informa- 
tion, and were blind to each other's diagnoses. The 
two assessments were scheduled with minimal time 
lag between them. All assessment sessions were video- 
recorded, except those for three participants who did not 
want to be recorded. 

To ensure that the clinical intake interviews were in 
fact appropriate to serve as expert opinion for validating 
the MINI, inter-rater reliability was computed. Eleven 
(27%) of the recorded clinical interviews were reassessed 
by a clinical psychologist who was familiar with the 
patient population and skilled in NSL. He received 
the referral information and made assessments based 
on the videotaped interviews, but was blind to the 
diagnoses given in the assessment. The same proce- 
dures were used to compute inter-rater reliability for 
eight (20%) of the MINI interviews. The patients were 
drawn randomly, but because the psychologist had to 
be blind to the person's diagnoses, the same patient 
could not be reassessed twice. To prevent resampling, 
the patients whose intake interviews had been used 
for reassessment were excluded before the random 
subsample was drawn. 
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Ethics 

Study inclusion required informed consent. Participants 
were informed that they could withdraw from the study 
at any time and have their videos erased, without 
any consequences for their treatment. Any participant 
who might experience additional burden as a result of 
participation in the study would be offered appropriate 
support and counselling. The study was approved by 
the Regional Committee for Medical and Health Research 
Ethics (REK) and the Norwegian Data Protection Authority 
(NDPA). 

Statistical analysis 

SPSS version 20 was used to analyse the data. Cohen's 
kappa was calculated to assess inter-rater reliability and 
to estimate the validity of the diagnoses assessed by the 
MINI. According to Shrout [28], kappa values above 0.80 
indicate substantial agreement, 0.61-0.80 moderate agree- 
ment, 0.41-0.60 fair agreement, 0.11-0.40 slight agreement, 
and below 0.10 virtually no agreement. Approximate 95% 
confidence intervals were generated by multiplying the 
standard error of kappa by 1.96. The observed agreement 
was the frequency of two raters' agreement on whether a 
disorder was present [29]. 

Results 

The mean length of the MINI interview was 58 min 
(range 32-88). The most prevalent diagnoses assessed by 
expert opinion were major depressive disorder (n of diag- 
noses = 22; 54%) and anxiety disorder (n of diagnoses = 12; 
29%). No participant met the criteria for mania, agorapho- 
bia, obsessive compulsive disorder, anorexia nervosa or 
bulimia nervosa. 

Twenty-eight participants (68%) received a diagnosis by 
both expert opinion and the MINI, with an average of 1.2 
diagnoses according to expert opinion and 1.9 according 
to the MINI. Thirteen participants (32%) did not meet the 
criteria for any diagnosis on the MINI. Two of them did 
not receive a diagnosis by expert opinion — four were con- 
sidered to have major depressive disorder — and seven 
were diagnosed with disorders of psychological develop- 
ment, mild mental retardation, personality disorder or 
disorder of severe stress and adjustment, which are included 
in ICD 10, but not in the MINI. 

There was agreement about 49 (64%) of the 77 diagnoses 
given to the 41 participants. 

In five conditions (6%) MINI gave no diagnosis and 
expert opinion assessed major depressive disorder, and 
in 13 conditions (17%) with a MINI diagnosis expert 
opinion assessed "no diagnosis". Nine conditions (12%) 
were assigned a diagnosis included in the MINI by the 
MINI interview and a diagnosis not included in the MINI 
by the clinical interview. Last, there was major disagree- 
ment about one condition (1%), which was diagnosed as 



drug dependence by the MINI and as major depressive 
disorder by expert opinion (Additional file 1: Table SI). 

Table 3 shows the concordance between the MINI and 
expert diagnoses. The participants' conditions covered a 
wide range of diagnoses, and the two gross diagnostic 
categories of depressive disorders and anxiety disorders 
occurred in numbers sufficient to conduct kappa analyses. 
Four diagnoses were assigned to fewer than 5% of the 
participants. The point estimate of Cohen's kappa was 
0.46 for depressive disorders, indicating fair agreement 
according to Shrout [28], and 0.72 for anxiety disorders, 
indicating moderate agreement. The observed agreement 
between the raters was 73% and 88%, respectively (Table 4). 

Twenty-six per cent of the participants underwent the 
two diagnostic interviews during the same day. The 
median number of days between the two interviews was 
19 (range 0-115). The point estimate of Cohen's kappa 
for depressive disorders was 0.46 for all participants, 
0.50 for those who underwent the two interviews within 
19 days, and 0.42 for those who waited longer than 
19 days. For anxiety disorders the corresponding kappas 
were 0.72, 0.77, and 0.63. 

Eleven participants who were diagnosed with a total of 
17 conditions were used in the inter-rater reliability 
analysis of the diagnoses assessed by expert opinion; 
there was agreement about 11 conditions (65%). There 
was disagreement about six conditions (35%) concerning 
co-morbidity. For two conditions (12%), the initial expert 
opinion gave a diagnosis and the second rater did not; in 
another two conditions (12%), the reverse occurred. There 
was also major diagnostic disagreement about two 
additional conditions (12%) of co-morbidity. In some 
cases of disagreement, the second rater's expert opinion 
diagnosis was in agreement with the MINI diagnosis and 
in conflict with the expert opinion. For instance, in one 
case, both the second rater and the MINI identified drug 
use, whereas the first expert diagnosed personality disorder 
(Additional file 2: Table S2). 

Cohen's kappa was computed for the two gross diagnostic 
categories "depressive disorders" and "anxiety disorders". 
The point estimate of Cohen's kappa in the analysis of the 
open clinical interview diagnoses was 0.81 for depressive 
disorders, indicating substantial agreement; the observed 
agreement was 91% (Table 5). For anxiety disorders, the 
point estimate of kappa was 0.44, indicating fair agreement, 
whereas the observed agreement was 73%. 

In the inter-rater analysis of the diagnoses assessed by 
MINI, eight participants were diagnosed with 13 condi- 
tions; there was agreement about nine conditions (69%). 
In two conditions (15%) diagnosed by the initial MINI 
team as alcohol dependence and major depressive dis- 
order, the second rater found no diagnosis. One condition 
(8%) was assessed as a dissocial personality disorder by 
the initial MINI team and as a psychotic disorder by the 



0hre ef al. BMC Psychiatry 2014, 14:148 
http://www.biomedcentral.com/1471-244X/14/148 



Page 7 of 10 



Table 3 Concordance between the Norwegian sign language version of the MINI and expert opinion, all patients' 
diagnoses (N = 41) 





MINI assessment 






Expert opinion 


- + 

+ - + + 






Disorders assessed in the present study (sorted as in MINI) 




Cohen's kappa (95% CI) 


Observed agreement 



Major depressive disorder 


17 


3 


0.51 (0.26-0.77) 


76% 




7 


14 






Dysthymia* 


37 


3 


0.38 (0.00-0.91) 


93% 




0 


1 






Panic disorder 


31 


7 


0.39 (0.07-0.72) 


83% 




0 


3 






Social anxiety disorder 


34 


3 


0.69 (0.37-1.00) 


93% 




0 


4 






Posttraumatic stress disorder 


37 


2 


0.36 (0.07-0.93) 


93% 


Alcoliol dependence & abuse 


1 

38 


1 
1 


0.79 (0.39-1.00) 


98% 




0 


2 






Substance abuse* 


37 


3 


0.38 (0.00-0.91) 


93% 




0 


1 






Psychotic disorders* 


40 


0 


1.00 (1.00) 


1 00% 




0 


1 






Generalised anxiety disorder 


37 


2 


0.00 (0.00) 


90% 




2 


0 






Antisocial personality disorder* 


39 


1 


0.66 (0.03-1.00) 


96% 




0 


1 







*Diagnoses occurring in less than 5% of the participants according to expert opinion. 



second rater, and one condition (8%) was diagnosed as 
major depressive disorder by the initial MINI team 
and as a dissocial personality disorder by the second 
rater (Additional file 3: Table S3). 

For the MINI assessments, the point estimate of 
Cohen's kappa was 0.71 for depressive disorders, indicating 
moderate agreement, and the observed agreement was 
88% (Table 6). For anxiety disorders, the agreement 
was complete (kappa = 1.00, observed agreement = 100%). 
Four instances of inter-rater disagreement mirrored the 
disagreement between expert opinion and the MINI 
diagnoses in the validation analysis. 

Discussion 

The MINI and expert diagnoses in deaf and severely 
hard-of-hearing signers referred for psychiatric evaluation 
show fair to substantial agreement. For validation against 
expert opinion, the kappa estimates in the present study 
are higher than those of Otsubo et al. [15] but lower than 
those of Kadri et al. [16]. Pinninti et al. [19] and Bobes 
[13] also used expert opinion for validation but did not 
offer kappa values. 



The kappa scores were lower than those of studies that 
have validated MINI against SCID-P [5,15,17] and CIDI 
[4]. This is in line with other validation studies [13,15,19]. 
The kappa values in validations of the MINI against expert 
opinion were generally lower than when the MINI has 
been validated against a structured diagnostic instrument 
such as SCID-P or CIDI [4,5,15,17]. Only Kadri et al. [16] 
report relatively high concordance with expert opinion. 
The open clinical interview and the structured MINI 
interview represent different approaches to assessing 
mental disorders, and complete agreement therefore 
cannot be expected. Structured psychiatric interviews are 
designed to elicit information about core symptoms of a 
set of mental disorders, and the concordance of two 
structured interviews is therefore likely to be higher than 
for a structured and an open interview. The diagnostic 
reflections of experts using an open diagnostic interview 
are likely more diverse than the focused questions and 
selected list of disorders in the structured interviews. 

The concordance of diagnoses was examined by two 
different methods because the sample size limited the 
use of kappa statistics. A broad range of diagnoses was 
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Table 4 Concordance between the Norwegian sign 
language version of the MINI and expert opinion, all 
patients' diagnoses (N = 41) 

MINI 

assessment 



Expert opinion 




- + 








+ - 


+ + 






Psychiatric disorders 






Cohen's kappa 


Observed 








(95% CI) 


agreement 


Depressive disorders^ 


14 
6 


5 

16 


0.46 (0.19-0.73) 


73% 


Anxiety disorders^ 


26 


5 


0.72 (0.4-0.94) 


88% 




0 


10 







^Depressive disorders include major depressive disorder, current; major 
depressive disorder, recurrent; and dysthymia. 
^Anxiety disorders include panic disorder, social anxiety disorder, 
post-traumatic stress disorder, and generalized anxiety disorder. 



found, some with very low frequency. Cohen's kappa is a 
conservative indicator and functions best with larger 
samples. It is unlikely to be reliable when a diagnosis is 
assessed in fewer than five per cent of a sample [30]. Only 
"depressive disorders" and "anxiety disorders" were identi- 
fied frequentiy enough to assess reliable kappa scores. 
Matrixes were therefore drawn to show the distribution of 
diagnoses assessed with the two diagnostic methods and in 
the inter-rater reliabOity procedure. This concordance ana- 
lysis indicates disagreement in approximately one-third of 
the diagnoses, with most disagreements related to the 
participants' second diagnosis (co-morbidity). This finding 
applies to the validation of MINI versus expert opinion as 
well as to the inter-rater reliability of the expert diagnosis. 

The main question in the present study is whether the 
NSL version of the MINI functions in the same manner 
as do other versions of the MINI when compared with 
expert opinion. Some of the lack of agreement between 
the two approaches may be caused by differences in 

Table 5 Inter-rater reliability estimates of diagnoses 
assessed by the initial clinical intake interview and by 



the second rater (N 


= 11) 






Expert opinion 


Second rater 

- + 

+ - + + 






Psychiatric disorders 




Cohen's kappa 
(95% CI) 


Observed 
agreement 


Depressive disorders^ 


4 1 

0 6 

5 1 
2 3 


0.81 (047-1.00) 


91% 


Anxiety disorders^ 


0.44 (0.00-0.97) 


73% 



^Depressive disorders include major depressive disorder, current; major 
depressive disorder, recurrent; and dysthymia. 
^Anxiety disorders include panic disorder, social anxiety disorder, 
post-traumatic stress disorder, and generalized anxiety disorder. 



Table 6 Inter-rater reliability estimates for diagnoses 
assessed by the Initial MINI assessment and by the second 
rater (N = 8) 





Second rater 






MINI assessment 




- -1- 








-1- - 


-1- -1- 






Psychiatric disorders 






Cohen's kappa 


Observed 








(95% CI) 


agreement 


Depressive disorders' 


5 
1 


0 
2 


0.71 (0.21-1.00) 


88% 


Anxiety disorders^ 


5 


0 


1.00 (1.00) 


1 00% 




0 


3 







^Depressive disorders include major depressive disorder, current; major 
depressive disorder, recurrent; and dysthymia. 
^Anxiety disorders include panic disorder, social anxiety disorder, 
post-traumatic stress disorder, and generalized anxiety disorder. 



diagnostic options. The MINI includes a limited number 
of diagnoses, and assessors will have to conform to those 
diagnoses or assign "no diagnosis". Relying on the MINI 
for diagnostic assessment may therefore influence the 
direction of the evaluation, as clinicians attempt to com- 
ply with the diagnoses at hand. Clinicians who conduct 
an open interview have a wider range of diagnoses to 
consider, which again may shape the direction of their 
clinical line of thought and the resulting diagnosis. The 
fact that the kappa values in this study were in the lower 
range compared with other validation studies may also 
be partiy explained by the presence of less severe disorders 
and fewer symptoms among the outpatients in this study 
than among the inpatients and acute ward patients 
assessed in most other studies (see Table 1). 

The MINI indicated a larger number of co-morbid 
conditions than did the expert opinion. Similar results 
have been reported in other studies [15,19] and suggest 
that the structure of the MINI may make the clinician 
more attentive to symptoms related to disorders add- 
itional to the main diagnosis, and that this attentional 
shift reduces diagnostic shadowing. This interpretation 
speaks in favour of including the MINI in routine assess- 
ment of signers, as is practiced for hearing patients. On 
the other hand, assessing severe and complex mental 
conditions by diagnostic categories exclusively may also 
increase the probability that symptoms are overlooked. 

The MINI was originally developed to meet the need 
for a structured assessment instrument that could be 
administered in minimal time. In the present study, the 
average MINI assessment required almost three times 
the estimated "less than 20 minutes" [8]. There may be 
several reasons for this. The interviews were conducted 
by two therapists, a method requiring collaboration and 
coordination, which may account for part of the extra 
time used. Furthermore, as was pointed out by Black and 
Glickman [3], many deaf patients with mental disorders do 
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not have full mastery of sign language and may struggle in 
expressing themselves and in comprehending what is being 
communicated to them, and this may lead to many repeti- 
tions. Finally, the interviewer cannot simultaneously take 
notes and communicate with the patient, and consequently 
the time needed for taking notes will add to the total inter- 
view time [12]. However, the reason for including the 
MINI as an assessment instrument for deaf and severely 
hard-of-hearing persons was to improve consistency, not 
to reduce assessment time. 

The two most frequent diagnostic categories in the 
present study were "depressive disorders" and "anxiety 
disorders". This result is similar to that of studies of psy- 
chiatric patient populations in general [8,31,32] as well 
as studies of mental disorders in signing deaf and 
hard-of-hearing patients [3,33]. To ensure that the open 
clinical intake interview would serve as expert opinion for 
the validation of the NSL version of the MINI, these two 
categories were used to assess inter-rater reliability. A 
second rater re-diagnosed the participants from video- 
recordings of the original assessment interviews, a common 
procedure in reliability studies [30]. To our knowledge, this 
approach has not been applied before in research with deaf 
and severely hard-of-hearing persons. It poses quite a few 
challenges to the second rater, who is forced to follow "the 
diagnostic route" of the clinician who conducted the initial 
assessment. This is no problem if the second rater agrees 
with the first, but second raters cannot collect additional 
information or follow alternative diagnostic routes when 
they disagree with the assumption of the original assessor. 
We do not know whether this issue influenced the 
concordance estimates in the present study. 

The average time span between the two assessments was 
longer than reported in other validation studies of the 
MINI. Twenty-six per cent of the participants went through 
both interviews the same day, which may seem optimal. 
However, participation was voluntary and cancellations and 
postponements of appointments were rather common. 
Furthermore, the participants' symptoms may have influ- 
enced their daily functioning and ability to keep appoint- 
ments. The centralised service for this geographically 
scattered patient population is likely to have bearing on 
the time between assessments. When participants lived far 
from the outpatient clinic and the assessments could not 
be made on the same or consecutive days, the result was 
sometimes a long interval between assessments. However, 
kappa values for assessments made with short and long 
intervals were quite similar. 

Limitations 

There are some limitations to this study. First, the relatively 
small sample size, broad range of diagnoses, and low 
numbers for some diagnoses limit the reliability of 
kappa statistics. Another disadvantage of kappa statistics 



is that scores indicating adequate or inadequate reliability 
may be considered arbitrary. To be stringent, the rather 
strict classification scheme suggested by Shrout [28] 
was used. 

Conclusion 

The kappa scores in the present validation of the NSL 
version of the MINI and the observed agreements with 
expert opinion do not differ substantially from those of 
other validation studies. The NSL version appears to be 
a reliable diagnostic interview for assessing mental disor- 
ders in signing persons, provided that it is used by profes- 
sionals with appropriate sign language skills and knowledge 
about the patient group. The results of the present study 
therefore encourage translation of the MINI into other sign 
languages. 
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